Qualitative assessments of mobile communications network performance are essential for network operators to ensure that an offered service meets certain requirements. Such information is particularly useful in determining quality of service (QoS) in mobile communications network standards such as General Packet Radio Service (GPRS), Enhanced Data Rates for GSM Evolution (EDGE), Universal Mobile Telecommunication System (UMTS), etc.
While statistics regarding the overall performance of a mobile data network or portions thereof provide useful performance metrics in some circumstances, the most pertinent data relates to user perceived application level performance. The analysis of application level performance can be quite complex as application level performance may be dependent on factors such as various network components and their performance (e.g., packet/signalling delays in the Serving GPRS Support Node or the Base Station Controller), radio protocol performance, transmission/application level protocols (e.g., TCP or WAP), radio conditions present within a cell, as well as the mobile equipment utilized. Performance metrics correlated to the type of mobile terminal type, such as multi-slot capability, packet processing time, software/hardware speed, protocol implementation, and radio signal processing are particularly useful as the type of mobile terminal has a significant effect on application QoS.
Conventional GPRS network systems utilize an Operation and Maintenance System for passively monitoring statistical performance indicators. These indicators, which comprise counters and statistics about events in different parts of the system (such as cells, Base Station Controllers, or GPRS Support Nodes) are used for monitoring network performance and the supervision of network resources. While these statistics contain aggregated data regarding the overall performance of the network (through indicators such as network equipment performance, radio protocols, radio condition variations by geographical location, end user equipment performance, etc.), such data is not suitable for characterizing specific device groups (such as the performance of a specific type of mobile terminal). Furthermore, these statistics only relate to lower protocol layers (e.g., radio protocols, cell resources, TBF allocations, etc.), rather than providing metrics regarding application level performance.
In some arrangements, it is possible to correlate QoS measurements to individual users on the Gi interface located between the Gateway GPRS Support Node and the external Public Data network and on the Gb interface located between the Serving GPRS Support Node and the Packet Control Unit. These correlations identify users through their International Mobile Subscriber Identity (IMSI) or Mobile Station International ISDN Number (MSISDN). While these statistics can be useful in analyzing individual usage statistics, the IMSI and MSISDN are associated with the Subscriber Identity Module (SIM) card, which may be used in connection with multiple types of mobile terminals (and so one cannot accurately assess performance by mobile terminal type).
Currently, there are some techniques that generate application performance statistics and benchmarks relating to unique mobile terminal types, but these techniques are not feasible for widespread adoption. For example, performance metrics may be generated through active measurement of mobile terminal performance (e.g., TEMS Investigation). When implemented for stationary tests, the mobile terminal remains at a fixed location, and for drive tests, the mobile terminal may be moved around during the test period. This approach is limited in that in order to get a statistically relevant amount of data, a large number of measurements must be conducted at different locations, and these measurements must be repeated for new applications and for each new mobile terminal as it becomes available. Furthermore, with this methodology, user data traffic is generated solely for the purpose of performance measurements, which often results in artificial measurements that do not accurately reflect typical mobile terminal usage.
Performance statistics for benchmarking may also be generated through passive techniques by capturing user data packets in the network to reconstruct the application or session level protocol conversation for mobile terminals. For example, a device within each mobile terminal may record various transaction statistics of interest which are periodically transmitted to a central interface unit for collection and evaluation. However, it will be appreciated that such an arrangement would unnecessarily burden the communication network by consuming bandwidth and would require the cooperation of each mobile terminal manufacturer for implementation.
It may also be possible to extract performance statistics from the Serving GPRS Support Node as it maintains the location of an individual mobile terminal in the Mobility Management context and in the Packet Data Protocol context for mobile terminals in STANDBY and READY states. While these context fields include the International Mobile Equipment Identity (IMEI) which can be used to determine the identity of the mobile terminal, such an arrangement would require significant changes in the software for the communications network and would also require each mobile terminal vendor to make certain changes to their mobile terminals.
Consequently, it can be appreciated that there is a need for an improved technique for assessing the performance of mobile communications terminals on a type-by-type basis.